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1 .  Introduction 

The  Multi    Knapsack     (MK)  problem  is  defined  as  follows, 

Let  each  item  j  (j=1,2,...n)  require  a  certain  amount  of 

space  a  .  of  knapsack  i  (i=1,2,...m)  and  let  c.  denote 
i]  3 

the  profit  of  having  item  j  in  the  knapsacks.  Let  x.  be 
a  zero-one  variable,  taking  value  one  if  item  j  is 
included  in  the  knapsacks,  and  zero  otherwise.  The  capa- 
cities of  the  knapsacks  are  denoted  by  b. ,  b^,...b  . 
This  leads  to  the  following  optimization  problem: 


n 

max 
j 


y  c .X . 


n 


s.t.   y   a.^x.  <  b.         (i=1,2,...m)  (MK) 


t.   y   a.  .: 


X.  e  {0,1  }  (j  =  1 ,2, . . .n) 


MK  has  been  used  to  model  problems  in  the  areas  of 
scheduling  and  capital  budgeting  /10/.  The  problem  is  known 
to  be  NP-hard  /6/;  it  is  a  generalization  of  the  standard 
knapsack  problem  (m=1).  MK  can  be  solved  by  a  polynomial 
approximation  scheme  /5/,  but  a  fully  polynomial  one 
cannot  exist  unless  P=NP  /9/, 

In  this  paper,  we  are  interested  in  analysing  the  be- 
haviour of  the  optimal  value  of  MK  with  respect  to  changing 
capacities  of  the  knapsacks.  More  specifically,  we  would 
like  to  obtain  an  expression  that  represents  the  optimal 

value  as  a  function  of  b.  (i=1,2,...m)  over  a  range  of 

1 


problem    instances. 
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A  stochastic  model  for  the  problem  parameters  c., 
a    (1=1, 2,... m,  j=1,2,...n)  will  show  that  the  sequence  of 

optimal  values,  properly  normalized,  converges  with 
probability  one  (wp1 )  to  a  function  of  the  b.s  as  the 
number  n  of  items  tends  to  infinity  while  the  number  m  of 
knapsacks  is  fixed. 

A  number  of  probabilistic  analyses  of  approximate  algo- 
rithms for  the  knapsack   problem  (m=1 )  have  been  conducted 
in  the  past  (/I/,  /2/,  /7/,  /11/).  So  far,  the  optimum 
value  has  not  been  asymptotically  characterized  as  a 
function  of  the  capacity  b  of  the  knapsack.  A  similar 
comment  applies  to  the  probabilistic  analysis  developed  in 
/5/  for  MK  under  a  stochastic  model  less  general  than  the 
one  we  consider  here.  In  fact,  our  only  requirement  will  be 
absolute  continuity  of  the  distributions  involved;  the 
results  are  unusually  robust. 

The  main  result  in  Section  3  is  the  asymptotic  charac- 
terization of  the  optimum  value  as  a  function  of  b. 

'^  1 

(i=1 , 2 , . . .m) .  This  characterization  is  obtained  by  showing 
that  the  optimal  values  of  the  LP-relaxation  of  MK  (denoted 
by  MKLP)  have  a  regular  limiting  behaviour  holding  wpl . 
The  fact  that  the  optimal  values  of  MK  and  MKLP  are 
asymptotic  to  each  other  follows  from  a  result  in  Section  2. 

Three  other  relevant  results  are  obtained.  To 
characterize  the  integer  optimal  solution  itself,  we  shall 
prove  that  the  zero-one  vectors  in  a  certain  sequence  are 
optimal  solutions  of  MK  infinitely  often  (i.o.).  We  also 
show  that,  relative  to  the  optimal  integer  solution,  at 
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least  one  constraint  has  a  slack  variable  whose  expected 
value  equals  zero  i.o.  .  Finally,  we  prove  that  the  se- 
quence of  optimal  dual  multipliers  of  MKLP  also  has  a 
regular  asymptotic  behaviour,  converging  to  a  limit  wp1 . 

In  Section  4,  the  results  obtained  are  applied  to 
problem  MK  for  the  cases  that  m=1  and  m=2  and  for  specific 
distributions  of  the  parameters   so  as  to  obtain  explicit 
formulae  for  the  optimal  value  of  MK,  They  are  depicted 
in  Figures  1  and  3. 

Finally,  in  Section  5,  an  approximation  algorithm  is 
proposed.  It  generalizes  the  greedy  heuristic  for  MK  (cf. 
/3/)  and  will  be  shown  to  be  asymptotically  optimal  wpl . 


$n 
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2 .  Stochastic  model  and  upper  and  lower  bounds 

Let  us  assume  that  the  profit  parameters  c.  (j=1,2,...n) 
are  independent  identically  distributed  (i.i.d.)  random 
variables  (r.v.),  defined  over  a  bounded  interval  in  E  with 
common  distribution  F  .  Analogously,  for  each  i=1,2,...,m, 
the  requirement  coefficients  a..  ( j  =  1  ,  2  ,  . . . , n)  are  assumed 

to  be  i.i.d.   r.v.'s   with  common  distribution  F.  over  a 

1 

bounded  interval. 

In  the  sequel,  r.v.'s  will  be  underlined.  Without  loss 

of  generality  the  interval  on  which  c.,  a  .  are  defined 

-J      -13 

will  be  assumed  to  be  [_0,1_[  .  We  further  assume  that  the 

distributions  F  ,  F.  ( i=1  ,  2  ,  .  . .  ,m)  are  absolutely  continuous 

c    1 

and  that  the  corresponding  densities  f  ,  f.  (i=1 , 2 , . . . ,m) 

are  continuous  and  strictly  positive  over  (0,1). 

It  is  reasonable  to  assume  that  the  capacities  b.  grow 

proportionally  to  the  number  of  items.  Specifically,  let 

b.=nB.  (i=1 ,2, . . . ,m)  for  66V  =  (6:  0<6.<Era.  1,  (i=1 , 2 , . . . ,m)} 
11  1    ' — il-' 

As      remarked      in     /11/  ,    the   i-th  constraint 

would  tend  to  be  redundant  if  S . 5E  fa . .  |  ,  in  the  sense 

1    ' — il-i 

that  it  would  be  satisfied  even  if  all  items  are  included, 
with  probability  tending  to  one  as  n^°°. 
Define 

n       n 

y  C.X.I  y  a . 

=1  -^-^1=1 


z  =max{  y  c.x.l  y  a..x.ine.  {  i  =  l ,  2  ,  .  .  .  ,m)  ,  x  .  s  j;  0  , 1  M  j  =  l ,  2  ,  .  .  .  n 
-n      ^^-3  3  '  /,-iD  D 


to  be  the  optimal  value  of  MK  and 
n         n 


LP 
z   =max 
— n 


{  y   c.x.l  y  a..x.$n6.  (i=1 , 2 , . . . ,m) ,  O^x.^1  (j=1,2,...n) 


to  be  the  optimal  value  of  MKLP . 


5  - 


■  f  ■  -J  r  ■  -f 

/  ... 


By  definition 


T,P 

I 

z 

> 

z 

n 

n 

(2.1) 


for  each  realization  of  the  random  parameters. 

To  bound  z   from  below  the  following  lemma  can  easily 
— n 

be  established   : 


I       LP 
LEMMA  2.1:     z    ^   z   -m . 
n       n 

Proof  :   The  optimal  solution  of  MKLP  has  at  most  m 
basic  variables  that  are  fractional.  Rounding  down  these 
values  yields  a  feasible  integer  solution  which  is  a  lower 

bound  on  z   .  The  value  corresponding  to  this  rounddown 

"  LP 

solution  is  given  by  z    decreased  by  at  most  m»max{c . }=m.l 
^       ^   n  J 
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3 .  Asymptotic  characterization  of  the  optimal  value 


In  order  to  derive  a  function  of  b.  (i=1  ,  2 , . . . ,m)  to 

LP 
which  _z    is  asymptotic  wp1  ,  we  consider  the  Lacvanaean 

relaxation   of  the  problem  MKLP ,  defined  as 


m 


n 


w  (X)  =  max  {  y   X .b.+  y 
— n  .,11^ 


i=1 


m 


c  .  -  y  X  .  a  .  . 
-1  .  .  1-11 


j  =  n  ■"    i=1 


J 


O^Xj<:i  ,(j  =  1,2, 


,n)} 


It  is  well  known  that,  for  every  realization  of  the 
)chastic  parameters,  th( 
the  region  X$0.  Moreover, 


stochastic  parameters,  the  function  w  (X)  is  convex  over 

n 


min   w  (X 
X^O    " 


LP 


=  z 


n 


(3.1) 


Let  X"  be  a  vector  of  multipliers  minimizing  w  (X) 
— n  — n 


Define  the  r.v.'s 


m 


x^(X) 


1    if  c  .-  y  X  .a.  .  5:  0 


i=1 

0    otherwise 


(j  =  1  ,  2  ,  .  .  .n)  ■ 


Then 


w 


m        n 

(X)  =   y  x.b.+   y 

1=1       j=1 


m 


c  .  -  y  X  .  a  .  .    X  .  ( X ) 
-D  ,i^  1-13    -D 


Define  also 


1  m  n 

L  (X)  =-  w  (X)  =    y  X. 6.+  -    y 

1=1  3=1 


( 


-n 


n  — n 


m 


I. 


c  .  -  y  X  .  a  . 

-3  ^t^    1-13  I  -3 

) 


x^(X) 


We    first   prove  a  preliminary  lemma  that  establishes 

the  almost  sure  convergence  of  L  (X)  to  a  non-random 

— n 
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function  L(X),  defined  by 


m  m 

L(X)  =  I    A.6.+E  ex  (X)  -  I    X.    E   a.x  (A)  . 

..11    —        .  .  1    ~i- 
1=1  1=1 

L  L 

Here   E  ex  (A)  and  E  a.x  (A)  are  used  to  denote  respecti- 
vely the  common  values  of  E  fc.x.  (A)  |  and  E  ra..x.(A) 1 

L  -3-3    J^       ^-xj-3         J 

( j=1 , 2 , . . . n) .  We  shall  also  write  E  ax  (A)  for  the  vector 

(E  a,x  (A),  E  a_x  (A)  ...,  E  a  x  (A)). 
—  1—        —2—  — m— 

We  observe  that  the  r.v.'s  {c.x.(X)}   (j=1,2,...n)  are 

L 
i.i.d.,  as  well  as  the  r.v.'s   {a..x.(A)}  (j=1,2,...n)  for 

-i]-j 

any  16  {l,2,.,.m}.  This  property  will  be  used  throughout 

the  paper,  sometimes  without  being  explicitly  mentioned.  In 

particular,  it  can  be  used  to  apply  the  strong  law  of 

large  numbers  to  the  sequence  L  (X)  and  obtain  the  following 

— n 

result. 


LEMMA  3.1:  For  every  A^O,  L  (X)  ^^^>  L(X)  .■ 
— n 

The  next  two  lemmas,  whose  proofs  are  given  in  the  Ap- 
pendix, describe  some  properties  of  the  function  L(X)  which 
will  turn  to  be  useful  in  the  sequel. 

LEMMA  3.2:  L(X)  is  twice  continuously   dif f erentiable  and 
strictly  convex.  For  each  66V  it  has  a  unique  non-zero 
minimum  X"=X"(B)  over  the  region  X5O.  The  gradient  is  given 

by 

Vl(X)  =   3-E  ax^(X)  .■ 

LEMMA  3.3;  For  X-  the  following  conditions  are  satisfied 
for  i=1 , 2 , . . .m: 
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(i)       A"V(3.-E  a.x-^(X")  )     =  0 
1   1    — 1— 

(ii)      E  a.x  (X")     <B.    .  ■ 

If  we  can  prove  that  L  (X'"')  >  L(,\"'),  then  through 

^  — n  — n  ^ 

(3.1)  L(A"')  would  provide  an  asymptotic  characterization  of 

1   LP 
—  z 
n  — n 

To  establish  this  convergence,  we  have  to  strengthen  the 

result  in  Lemma  3.1  from  pointwise  to  uniform  convergence 

wpl .  To  do  this,  we  shall  apply  a  theorem  from  /1 2/  which 

states  that  pointwise  convergence  of  a  sequence  of  convex 

functions  on  a  compact  subset  of  their  domain  implies 

uniform  convergence  on  this  subset  to  a  function  that  is 

also  convex. 

First,  we  have  to  show  that  the  minima   {X"}  (n=1,2,...) 

— n 

and  X"  are  contained  in  a  compact  subset  S  ^  R   .In  fact, 

we  shall  show  that  the  minimization  of  the  functions 

L  (X)  (n=1,2,...)  and  L(X)  over  R   is  equivalent  to  their 
— n  + 

minimization  over  the  set 

m 
S  ={X|  I       B.X.  ^1,  X.50   (i=1  ,2,  . . .  ,m)  }   . 
i=1 

LEMMA  3.4:   For  every  realization  of  the  stochastic  para- 
meters, the  functions  L(X)  and  L  (X)  (n=1,2,...)  attain 

n 

their  minimum  within  the  set  S. 
Proof:   it  is  easy  to  see  that 

1    ^ 
L  (0)  =  -    I         c.  ^1 

^   3  =  1    ^ 
For  any  other  value  of  A  it  holds  that 
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n        m 
L(X)=Xe+-   y   (c.-  I    X  .a.  .)    x^  {\)     >^    X    ^  . 

Hence,  for  every  A  with   X3  >  1/  we  have 

L  (A)  >  1  $  L  (0) 
n  n 

Convexity  of  L  (A)  implies,  together  with  the  inequalities 
above,  that  A"es,  n=1,2,...  .The  same  arguments  applied  to 
L(A)  show  that  A"eS.   ■ 


As  a  direct  application  of  Rockaf ellar ' s  theorem  10.8 
/1 2/,  we  then  obtain  the  following  result. 

wpl 
LEMMA  3.5  :  L  (A)  -^^>   L(A)  uniformly  on  S. 

We  are  now  in  a  position  to  prove  the  required  result. 

wp  1 
THEOREM  3.1:     L  (A  )  -^— >  L  ( A " )  . 
— n  — n 

Proof:   Uniform  convergence  wpl  on  S  can  be  written  as 

Pr{V£>0    3n  :  Vn^n    sup  II  (A)-L(A)|<e}  =  1   .   (3.2) 
o       o   ,^^  '-n         ' 

Aes 

It  is  easy  to  see  that 

II  (A")-L(A")|  ^  sup  |L  (A)-L(A)|     .  (3.3) 

'  — n  — n        '      '^  '  — n 

AGS 

The  combination  of  (3.2)  and  (3.3)  yields 

Pr{V£>0    3n  :  Vn^n   II  ( A " ) -L ( A " ) | < £ }   =  1 
o       o  '  — n  — n        ' 

which  proves  the  required  result.    ■ 

From  (3.1)  and  Theorem   3.1   it  follows  that 
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1    LP  Wp  1      ^  ,  .  :':  ,  /  o   /,  \ 

-  z    -^— >   L  ( X  )  .  (3.4) 


n  — n 


Moreover,  we  know  from  Lemma  2.1  that,  for  each  realization, 


1   LP    "^     11     1   LP 

-Z-  ^-Z^-Z      .  ,•^c^ 

nn     n     nnnn  (3.5) 


Combining  (3.4)  and  (3.5)  one  can  easily  establish  the 
following  theorem. 

1    I  Wp  1        ,,:':, 

THEOREM  3.2:    -  z   ^^— >   LA)   .   ■ 

n  — n 

This  latter  result  gives  the  required  as^^rmptotic 
characterization  of  the  optimum  of  MK ,  holding  almost  surely. 
Observe  that  L  ( A " )  is  actually  a  function  of  the  right- 
hand  side  b.   (i=1,2,.,.m)  implicitly  defined     by 
minimization  of  L(A)  over  S.  The  problem  of  deriving  a 
closed-form  expression  for  this  function,  or  at  least  of 
evaluating  it   numerically     for   different    values 
of  its  arguments  b.  (i=1,2,...m)  will  be  considered  in  the 
subsequent  Section  4   for  specific  distributions  F  ,F .(  i  =  1  ,  .  .  .  ,nil 
Whereas  Theorem  3.2  is  concerned  with  the  behaviour  of 

the  optimum  value  z   of  MK ,  the  following  result  says 

— n 

something  about  the  corresponding  optimal  solution. 

THEOREM  3.3  :   The  vector  x  ■■"  (  A  " )  =  (x^  (  A  "  )  ,  x^  (  A  "  )  .  .  .  x^  (  X  ")  ) 

—       —1       —2  ~n 

is  optimal  infinitely  often  (i.e.). 

Proof  :  Define  the  following  events:  Q  ={x  (A  )  is  not  optimal}, 

L   :.-  L 

R  ={x  (A  )  is  suboptimal}  and  T  ={x  (A")  is  not  feasible}, 
n  —  n   — 
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By  definition,  we  have  to  show  that  the  series  Zpr{Q  } 
is  convergent.  Obviously, 


Pr{Q  }  $  Pr{R  }  +  Pr{T  } 
n        n        n 


We  have 


n 


Pr{R  }  ^  Pr  {  3e:    —      )   c.x.(X  )<  -  z    -  e }= 
n  n  .  ^  ~3~3       n  — n 


r1   I    LP      ^  L,  .....  I    , 

Pr{-    z    -   )   c .X . (X  )  >e } 
n   -n    .^^  -:-: 


n 


Since  -  (  j      c.x.(A")  -  z   )  ^^^>  0,  the  series  ZPr{R  } 
n   .^^  -J-3        -n  n 

converges  for  every  £>0.  Moreover, 


1  L 

Pr{T    }    =   Pr{    3i,£>0:    B.-  -      I    a..x.(X")    <    -    e} 
n  in    .^^-iD-D 


1       "  L       • 

$mPr{6.    --      y      a..x.(X")     <-£}, 
1         n     .^^    -i]-D 


so  that  the  series  ZPr{T  }  is  also  convergent,  since  from 

n 

Lemma  3 . 3 

6.  -  -   )   a..x^(X")  ^>  6.-E  a.x^(X")  ^0  ( i=1  , 2 ,  .  .  .m)  .1 
1    n  ^^^  -ID-]  1    -1- 

As  a  conseguence  of  Lemmas  3.2,  3.3  and  Theorem  3,4  at 
least  one  constraint  will  be  binding  in  expectation  at  the 
integer  optimum  i.e.,  in  the  sense  that  the  expected  value 
of  its  slack  equals  zero  i.o. 
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However, this  property  is  not  true   in  general   for  all 
constraints     (cf.   our   comments  for  the  case  m=2  in 
Section  41 . 

To  conclude  this  section  a  stronger  version  of  Theorem  3.1 
will   be  proved  ,  which  extends  the  convergence  result 

from  the  sequence  of  minima  of  the  Lagrangean  L  (a)  to  the 

•.'.•  wp  T   " 
sequence  of  their  arguments,  showing  that  A   -^— >  X  .  This 

~n 

property  indicates    regular  asymptotic  behaviour  of  the 
sequence  of  optimal  dual  multipliers  _X'.  Moreover,  it  will 
be  useful  in  developing  the  approximation  algorithm  pro- 
posed in  Section  5. 

"  Wpl      ••: 

THEOREM  3.4:      X      -^— >  X  . 
• — n 

Proof;   A  Taylor     series  of  L(X)       around  ■  x"   can  be 
written  as 

L(X""')-L(X''')  =  (x""''-.\'"')  VL(X'"')  +  ^(x""'-x''')h;a  )  (x''"'-x'"')    (3.6) 
— n  — n  2  — n       n   — n 

where  H(X  )  is  the  Hessian  matrix  of  L(X)   evaluated  at  a 
n 

point  X   lying  on   the  line  segment  Qx.  ,  X  J   and  there- 
fore belonging  to  the  convex  set  S. 

Positive  definiteness  of  H(X)  for  every  X  (see  the 
Appendix)  implies  strict  positivity    of  its  eigenvalues. 
Let  a(X)  denote  the  smallest  eigenvalue  of  H(X),  and  let 
a=inf  a(X);  note  that  the  continuity  of  a(X)  implies  that  a>0- 

xes 

It  can  be  easily  shown  that,  for  any  n, 

^(X""'-X""')  H(X  )  (X'"'-X""')  >^  IU'"'-X""'||  ^    .  (3.7) 

2  — n        n   — n        2  "  — n 


-  13  - 

Moreover,  due  to  Lemma  3.3,  we  have 
-X"Vl(X")   =  0 


X"VL(X")  ^  0 
— n 


Combining  (3.6),  (3.7)  and  (3.8),  we  obtain  that 

L(X")-L(X")  >  ^   II  X"  -  X"||  2. 
— n  2     ~n 


Consequently,  to  prove  convergence  of  X_"  to  X",  it 

wp  1 

remains  to  show  that  L(X")  ^>  L  (  X  "  )  . 

— n 

Indeed, 


L(X""')  -L(X")|  ^  Il(X""')-L  (X'"')|  +  1l  (X""')-L(X")   , 
— n  — n   — n  — n     — n  — n 


and   |L(X")-L  (X")|  -£->  0  by  Lemma  3.5,  whereas 
'   — n   — n  — n  ' 

|L  (X")-L(X")|  -^>   0  bv  Theorem  3.1.B 
'  — n  — n        ' 


(3.8) 
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4 ,  Particular  cases:  m=1  ,  in=2 

The  actual  computation  of  E  ex  (X)  and  E  a . x  (X)  requires 

a  choice  for  a  specific  distribution  for  the  stochastic' 

parameters   of  MK .  in  this  section,  we  assume  that  the 

profit  coefficients  c.  as  well  as  the  requirements  a. .  are 
^  -3  ~i3 

uniformly  distributed  over  [[0,1^.  This  assumption  is 
essentially  the  same  as  in  /  5/,  /  7/,  /11/. 

We  first  present  and  discuss  the  results  concerning  MK 
with  one  constraint.  In  this  case,  straightforward  calcu- 
lations lead  to  the  following  formulae: 


E  ax  (X)  = 


1  A 

2  ~  3 


ex' 


if    \  i  ^ 


if   A  >  1 


-   -   —r  if   X  <  1 


E  ex  (X) 


3X 


if   X  >  1 
2 


L(X)  = 


^(p-  ^^  ^h  ''  ^'' 

6  X 


/6B 


1 

if  0<6<7 
6 


A  = 


3  .^  1   o   1 

-  -  36  If  g  ^6<  2 
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Therefore,  by  Theorem  3.2, 


'I  B  if  0<B<-^ 

1    I  Wpl      ,   :•: 

-1„^>  MA  )=       3„2   3    1  .11 

je  +  -6+  5  If  --<6<- 


The  graph  of  L(X  )   as  a  function  of  3   is  shovm  in 
Figure  1 . 

Notice  that  in  accordance  with  Leminas  3.2  and  3.3, 
E  ax  (a'")-B=0. 

In  the  case  that  m=2,  E  ex  (X)  ,  E  ax  (A)  and  E  a  x''^(X) 

^2" 
take  different  forms   over  different  regions  of  the  H 

plane.  We  will  describe  directly  the  corresponding  functions 

X  =X  (S)   defined  on  the  (6^,  B^)  plane,  referring  to 

Figure  2 . 

Define    the    following    regions: 

A   =       {  (3^,62)     :     S-i^B^/       B2^24B^} 

B  =     {(3^,62):   6^4b',   32^'  e,4} 

C    =      {(3^,32):     ^^>\.     324^1    -   \'    6,<J} 

D   =      {(3^,32):     3^^62,     B2>|b^-   I    ,     £^^32^:^} 

E  =   {(3^,62):  Bi^32.  B2>fB'  ,  6i^£2<l|>   . 


The  values  of  X   and  L(X  )  in  the  corresponding  five 
regions  where  B^  <  B_  can  be  obtained   by  exchanging  3   with 
3^  and  viceversa   in   the  formulae  given  below  . 


Region   A: 
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s  = 


^2 


242 


,        X. 


246. 


L(X    )     = 


V96^62 


Region   B: 


\ 

'       ^2 

1 

V6B2 

L(A")     =     ^ 

/^6       , 

3    "^2       x 

Region   C: 

V 

X-;  =  0 


x^  -  2  -  362 


l(a'"')   = 


3    2         3  1 

— B       +   —    P       +    — 
2'^2         2    "2         8 


Region    D: 


^i"   "   7(3662-48e.,+6) 


,     X2    =   -(366.,-48e2  +  6: 


L(X")     =    ^(-24B^-24e2  +  366.,62  +  66.,+662+    2' 
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Region  E  :  to  obtain  closed  form  equations  for  the  values 
X" ,    L(X")  for  B  lying  in  E,  an  equation  of  either  fourth 
or  eighth  degree  should  be  solved  explicitly.  Yet, 
numerical  evaluation  is  possible  through  the  direct  solu- 
tion of  the  problem  min  L(,\),  using  an  appropriate  non- 

,  Aes 
linear  programming  routine. 

A  picture  of  the  surface  L(X")/  defined  over  the 
(6^/6^)  plane  and  evaluated  either  analytically  or  nume- 
rically, is  presented  in  Figure  3. 

By  calculating  6 . -E  a.x  (A")  (i=1,2)  for  the  regions 
A,B,C  and  D  and  using  the  remark  at  the  end  of  Theorem  3.3, 
one  can  verify  that  in  A  and  D  both  constraints  are  i.o. 
binding  in  expectation  at  the  optimal  integer  solution, 
while  in  B  and  C  this  is  true  only  for  the  tighter 
constraint.  This  corresponds  to  intuition:  when  Q      is 
sufficiently  small  with  respect  to  6.,  as  in  B  and  C,  the 
first  constraint  can  be  disregarded, so  that  MK  with  two 
constraints  is  reduced  to  a  simple  knapsack  problem.  TO 
support  this  conclusion,  observe  that  the  values  of  A^  and 
L(A")  obtained  for  the  regions  B  and  C  are  identical  to 
the  corresponding  ones  derived  for  the  case  m= 1 . 

Similar  calculations  can  be  carried  our  for  m>3,  even 
though  in  these  cases  only  numerical  approximation  of  A" 
and  L(>.")  is  possible  for  many  values  of  6.  The  computation 
of  L(X)  and  VL{X)  can  be  seen  to  amount  to  integrating  the 
density  function  over  regions  defined  by  linear  inequalities, 
In  many  situations,  closed  form  expressions  for  these 
integrals  can  be  derived  in  principle. 
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5.  Probabilistic  analysis  of  an  approximation  algorithm 

The  algorithm  for  solving  the  MK  problem  considered 

in  this  section  belongs  to  the  family  of  generalized 

greedy    heuristics,  that  can  be  described  as  follows. 

Given  m  positive  weights    y^  ,  y„  ,  ...,y  /  order  the  items 

1    /        m 

according  to  decreasing  ratios 


c  . 

p.  =  ( j=1 , 2 , . . . ,n) 

J     m 


y    y.a.  • 

x=1 


The  generalized  greedy  heuristic,  denoted  by  G(y),  then 

select  items  according  to  this  order  until  the  next  item 

considered  will  violate  one  of  the  constraints  if  added 

to  the  knapsacks.  Let  z  (y)  indicate  the  value  of  the 

n 

solution  obtained  by  this  heuristic.  Obviously,  the  qua- 
lity  of  the  approximation  z  (y)  is  affected  by  the  choice 
of  the  weight  vector  y.  The  behaviour  of  G(y)  as  a 
function  of  y  will  be  analyzed  in  a  forthcoming  paper.  In 
particular,  it  is  possible  to  show  that  (under  a  non- 
degeneracy  assumption)  the  choice  y=A",  where  for  each 

problem  instance  \"    is  the  vector  of  optimal  dual  multi- 

n 

G 
pliers,  leads  to  a  solution  z  (.\'")  which  is  the  same  as 

n   n 

the  integer  round-down  of  the  optimal  solution  of  MKLP . 

A  drawback  of  this  choice  seems  to  be  that  y  depends  on 

the  particular  problem  instance  in  a  non-trivial  way.  Yet, 

in  light  of  Theorem  3.4  the  choice  y=X  where  \ '■'  is  defined  (as 

in  Section  3) to  be  the  unique  minumum  of  L(a) , seems  to  be  a  rea- 
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sonable  alternative,  at  least  in  a  probabilistic  frame- 
work. 

In  fact,   we  shall    show  that  the  generalized  greedy 
heuristic  corresponding     to   weights  y=A   is  asympto- 
tically optimal  wp1  with  respect  to  the  stochastic  model 
of  MK  introduced  in  Section  2. 

The  following  result   providing   bounds  on  the  proba- 
bility that  a  normalized  sum  of  r.v.'s  deviates  from  its 
mean   is  due  to  Hoeffding  /  8/   and  will  be  useful  in 
what  follows. 


LEMMA  5.1:   Let  X^,  X^,  ...X   be  independent  r.v.'s 
12       n         ^ 

taking  values  in  the  interval  10,11    and  having  finite 

-    -1     -  n    -1 

mean  and  variance.  Then,  for  0<t<1-  —  E   /,  X  . 


Pr 


n 


r 


n- 


J  =  1   ^ 


-  n    T  ^ 


t  ^  <   e 


-2nt 


(5.1) 


Pr 


Pr 


1  ( 


r  n 


n 


!■■<. 


n 


Lj  =  1  -'^  j  =  1 


I      ^j'^M^ 


<:   e 


n 


n 


-  n 


y    X  .-E 


-2nt 


t  U  2  e 


-2nt 


(5.2) 


(5.3) 


Define  the  r.v. 


n 


u   =  inf{u>0   I  J      a  .  .X  .  (uX")  $b.  (i=1  ,  2  ,  .  .  .  ,m)} 
n  .  .  —11—1        1 


j=1 
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Clearly,  jj   can  be  interpreted  as  the  value  of  the 
weight  p .  corresponding  to  the  last  item  included  in  the 
knapsacks  by  the  algorithm  G{X"). 

We  first  establish  a  convergence  result  for  the  se- 
quence of  r.v.'s   {u  }  . 

r-  .  Wpl 

THEOREM  5.1:     u   -^   1  • 
-^n 

Proof:    The  proof  will  be    split   in  two  parts.  We  first 

wp1  - 

show     that  u   >  u»  where 

-^n 

L 
y  =  inf{y^O:   E  a^x  (-X  )^3.   ( i=1 , 2 , . . .m) } . 

We  then   prove   that  '^  =  1  . 

Let  !_  (y)  and  I(y)  be  respectively  the  indicator  func- 
tions of  the  sets 

n 

a  . 


H   =  {y^O:  I      a.  .x^(yX'")  ^  b.    (1=1  ,  2  ,  .  .  .  ,m)  ^ 


j  =  1 
and 

Q    =    {y^O:  E  a.x  (yX")$  6-    (i=1,2,...m)}. 

wpl 

We  want  to  show  that   for  u>0,  I  (n)  ^>  I(u)-  Assume 

^  "    — n  ^ 

first  that  there  exists  at  least  one  index  k,  1<:k$m,  for 

L    :V  , 

which  E  ax  (yX  )>£,  ,  so  that  I(y)  =  0.   We  then  have 

K,  K 

that 
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1       ""  L         •- 

Pr    {ll    (y)-I  (y)  |>c}=   Pr{-      Y   a  .  .x  ,  ( y  X  ")  ^6  .  -  (  i=1  ,  2  ,  .  .  .  ,m)  } 

— n  n  .  v.~ij~i  1 

1       "  L  -••  L  •••  L 

=    Pr    {-      [    aj^jXj(y^    )-E  a^^x     (P^    ^^^k"^   -k-    ^^^    ^^ 


L         •'•      2 

-2n(B,  -E    a,  x     (yX")  )  ,^    ,. 

^    e  k        -k-      "^  ,  (5.4) 


where  the  last  inequality  is  derived  from  (5.2)  by  applying 
Lemma  5.1  to  the  i.i.d.  r.v.': 
Suppose  now  that  I(y)=1.  Then 


Lemma  5.1  to  the  i.i.d.  r.v.'s  ^-.■^■i'^iX")     (  j  =  1  ,  2  ,  .  .  .  ,  n) 

^3  3 


1   ""      L    - 

Pr{|l  (y)-I(y)l>£}  =  Pr  {  3k:  -   T  a,  .x.(yX  )  >  6,, } 

3  =  1 


n 


1          L    "        L    "  L    " 

^  n  Pr  {-   y  a   x.(yX  )-E  ax  (yX")>eT,-E  ax  (yx") 

3  =  1 

L    ■'•   2 

-2n  (B,  -E  a,  X  (yX")  )  ,^  ^. 

^  ne       k    -k-           ,  (5.5) 


again  by  Lemma  5.1. 

Combining  (5.4)  and  (5.5)  it  follows  that  the  series 

ZPr{|^  (y)-I(y)|>e}  is  convergent  for  yjiO  and  any  £>0,  so 

that  I  (y)  ^>  I(y)  . 
— n 

Turning  now  to  the  sequence  y  ,  we  have  that 
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Pr{|iJ  -ijl>e}  ^  Pr{y  -y>e}  +  Pr{y-y  >e} 
— n  — n  — n 


^  Pr{I(p+  f)-I  (y+  h-^]    +   Pr{I  (y-  f)-I(y-  f)  =  1} 
2   — n     Z  — n     2        2 

wpl 

for  any  £>0,   so  that   y   ^>  y. 

■  — n 

Now,  to  show  that  y=1  ,  we  recall  condition  (ii)  of  Lem- 
ma 3 .  3 , 


L   * 

E  a.x  (A  )  $  B  . 
—1—  1 


(i=1  ,  2  ,  .  .  .  ,m)  , 


(5.6) 


noticing  that  it  implies  that  y^l. Suppose  now  that  y<1. Since 
the  functions  E  a.x  (X  )   (i=1 , 2 , . . . ,m )  are  strictly  de- 
creasing in  each  component  X   (k=1 , 2 , . . . ,m  )  (see  the  Ap- 
pendix), y<1  implies  that  all  inequalities  in  (5.6)  hold 
strictly.  This  in  turn  implies   by  condition  (i)  of  Lem- 
ma 3.3   that  a'"=0,  contradicting  Lemma  3.2.   g 


We  are  now  in  a  position  to  prove  the  final  result. 


THEOREM  5.2:    z  ( X " )  ^^^>  L  (  X 
— n 


Proof:   By  definition  of  y_  ,  it  follows  that 

G         =^  ^  L  :V 

z(X)=       y       c.x.(yX). 


Let    ffi (y    ) =E 
—  — n      ^ 


£l^1     (Hn^"^  I    iin 


We   have    that,    for    b>0. 


Pr{  |-z^(>  "■'"') -L(X  ■)  \>z} 
n— ^ 


n 


Pr{  I  -       )     c  .X .  (y     A     ) -L(X")  I >e} 
n     >^-3-D    -n 
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^  Pr{ |-  I    c .x^(y  X")-  ^(£^) |+|^(y^)-L(A") I >e} 
n  ^^^  3  D   n 


-n 


-n 


1   " 


v<  Pr{|-  I    c.x.{^^x")-^iii^)\>-}+-Pr{\^{^^)-Lix")\>-]    .     (5.7) 


j  =  1 


Wpl        :': 

The  function  p{y)  is  continuous,  so  that  j^iu    )  — =— >  L(X  ) 
by  Theorem  5.1.  It  follows  that 


'.  c  . 


ZPr{  Icp  (u  )-L(a")  I  >-  }  <  0°. 
'—  -^n        '2 


(5.8) 


Moreover,  letting  F  (u)  be  the  distribution  of  u  ,  we 

n  ^  ■'=^n 

have  that 


■•j.c 


n 


Pr{  I-   y  c  .X  .  (u  x")-  (p  (u  )  I  >  -  }  = 


n  .^^-3-j    -n 


n 


Pr{  |-   Tex.  (yX")-  ^(y)  |>  ■^}  dF  (y) 


n 


D  =  1 


n 


2e     4 


(5.9) 


the  last  inequality  being  derived  from  (5.3),  since  Lemma 
5.1  can  be  applied  to  the  i.i.d.  r.v.'s  c.x.(yX"). 

Combining  (5.7),  (5.8)  and  (5.9),  one  finally  obtains 
the  required  result.   ■ 

Further  investigation  of  the  family  of  heuristics  G(y) 
seems  to  be  an  interesting  topic  for  future  research. 
Although  the  results  of  this  paper  carry  over  immediately 
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to  the  minimization  of  MK  (with  ^  replacing  $),  prelimi- 
nary investigations  suggest  that  the  heuristic  s  worst 
case  behaviour  for  these  two  models  differs  substantially, 
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■- ..  r: 


Appendix 

The  purpose  of  this  section  is  that  of  proving  Lemmas 
3.2  and  3.3.  To  this  end,  the  following  two  results  are 
useful . 

L  L 

LEMMA  A.I;  The  functions  E  ex  (A),  E  a.x  (A),  j=1,2,...m, 

are  continuously  dif f erentiable . and  strictly  decreasing 

with  respect  to  each  component  A,   (k  =  1,2,...m). 


Proof:  It  will  be  shown  that  the  partial  derivatives 

3Ea.x^(A) 

— TT exist  and  are  continuous  and  strictly  negative, for 

^\        ■ 
each  A   >_  0.  This  implies  the  required  result  for  Ea  x  (A); 

similar  arguments  can  be  applied   to  prove  the  Lemma  also 

for  Ecx  ( A ) . 

To  simplify  the  notations , assume  that  k=m  without  loss  of 

generality.  Let  a' = (a, , a. , . . .a    ),  and  define 

I   2      m-  I 

m 

D  =  {a,c  I  0<a<1 ,   0<c<1 ,   c>   ^  A.a.} 

i=1 

m-1 
D'=  {a',c|  0<a'<1,   0<c<1,  c>   T   A. a.} 
-----  i  =  1   "  " 

m-1 

1 


D  =  {a    0<a  <1,   A  a  <c  -   T   A. a. 

m    m   —  m—     m  m—      ^    i  i 

1=1 


m 

Let  F(a,,...a  ,c)  =  F  (c)*   il  F.(a.)  and  F'(a,,...a  ,c)  = 

1      m  c      .^11           1      m 

m-1  ^-' 

=  F  (c) •   n   F. (a.)  .  By  Fubini ' s  theorem,  we  have  that 

c            1   1  -^ 
i=1 


28 


Ea.x  (X)  =     a.dF 
J         3 

D 


a  .dF 

J      m 


D'     D 


m 


dF'  =     U(X  )dF'    (A.I) 
J  "^ 

D* 


where 


U(A  )  =       a .dF   = 
m      I     3   m 


m 


a  .dF  , 
3   m 


with 


m 


m 


m-1  m-1 

(c-   y   A.a. ) /X     if  c-  y   X.a.<X 
^         11    m         .  '- ,   1  1— 


i=1 


m-1 


i=1 

otherwise 


m 


For  0<X  <c-  y   X.a.,  the  function  U(X  )  is  constant, 

m    .  _    1  1  m 

hence  it  is  continuously  dif f erentiable . 

m-1 

For  X  >c-  y   X.a.,  by  the  absolute  continuity  of  F  ,  it 
m     ^    1  1    "^  ■'      m 

i  =  1 

follows  that  U(X  )  is  dif f erentiable ,  and 


dU 

dA 


m 


m 

r 


-  a.f 
3  m 


m-1 

c-   y    x.a. 
^      11 
i=1 


^ 


X 


I 


m 


m-1 
c-  y   x.a. 
i-1  '    ' 


/  j  ?^  m 


f 


-f 


m 


m-1 
c-  y   x.a. 
i'l   ^  ^ 


m 

^   ^   m-1      p 
c-  y   x.a. 
1=1      I 


X 


m 


3  =  m  . 


X" 


m 


(A. 2) 


dU 


For  each  X  >0,  the  function  -;—   is  continous  for 


m 


m 


all  (a' ,c)  in   D'    except    on      the  hyperplane 
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m-1 

c-  y   X.a.  =  X  .  Therefore,  the  function  U(A  )  is 
.  ^    1  1     m  m 

continuously  dif f erentiable  for  A  >0   almost  everywhere  in 

D'  with  respect  to  dF ' .  Since  D'  does  not  depend  on  A  , 

m 

it  can  be  concluded  / 1 3/  that 

9E  a.x  (A) 

— exists  and  is  continuous  for  A  >0.  In  addition, 

9  A  m 

m 


3Ea.x  (A) 


9  A 


dU 
dA 


dF 


m 


D' 


m 


dU 
dX 


dF'  , 


(A. 3) 


D' 
o 


m 


m-1 


where  D'={a',c:  0<a'<1,  0<c<1,  0<c-  7   A. a.  <  A  } 
o         —  —    —  —    —   .^^11—  m 

1  =  1 


since 


dU 

dA 


m 


0  on  D'-D' . 
o 


It  remains  to  be  shown  that  the  right  derivative  of 


Ea.x  (A)  in  A  =0  exists   and  is  equal  to  the  limit  of 
—J—  ra  ^ 

(A. 3)  for  A  ^0  .  To  see  this,  define 
m 

m-1 

D'  =  {a*  i  0<a'<1 ,    y   A .a.  <  1 }  , 
a         —  —     .^,11— 

1=1 


D'  =  {c  I  0<c<1  , 


D   =  {a  '  0<a<1  , 
a      '   —  — 


m-1 

0<c-  y   A. a.  <  A  } 
—    '-         1  1  —   m 
i  =  1 

m-1 

y   A .a.  <  1 }  . 

i=1  '    ^- 


We  have,  for  A  >0, 
m 


lim 

X  ^O' 
m 


3Ea.x  (X) 


9X 


m 


lim 

X  ^0^ 
m 


D" 
a 


dU 
dX 


dF 


dF' 
a 


(A. 4) 


D' 
c 


m 
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m-1 

where  F'(a')  =   n   F.(a!).  Applying  the  substitution 
a        .^11 
1=1 

m 

c=  y  X . a   to  the  inner  integral  in  (A. 4) ,  one  obtains  from 

■  -1  ■'■■'" 
(A. 2),  and  for  all  j=1,2,...m, 


1 


-^  dF 
dX     c 


m 


-  a  .a   f 
J  m   c 


^  m      ^ 

y  X.a. 

i=1  '    ' 


dF 


ra 


D' 
c 


Since  D   does  not  depend  on  X  ,  we  have 
a  ^        m 


lim 

X  -0  + 
m 


9Ea.x  (X) 


9X 


m 


/  lim 

-  X  ->0' 

D  m 
a 


-a  .  a  f 

J  m  c 


m 

y  X.a 
i=1  ^  ^ 


dF 


■/  -, 


a  f 
m  c 


m-1       j 

y    X.a.    dF   , 
■     .         11     a 
1  =  1 


(A. 5) 


m 
where  F  (a)  =   11   F  .  (a.  ) 
i=1   ^   ^ 


It  can  be  easily  verified  that  the  right  derivative  of 

Ea.x  (X)  in  X  =0   exists  and  is  equal  to  the  last  term  in 
-J-  m 

(A. 5) . 

To  conclude  that  Ea.x  (X)  is  strictly  decreasing  with 

respect  to  X  ,  observe  that 

m 

L 
dEa.x  (X) 

-J- 


.,  ,  is  strictly  negative  for  X  >0.  Indeed,  it  follows 

oX  J     3  fj^ 


m 


from  (A. 2),  (A. 3)  and  {A. 5)  that  this  derivative  is  equal, 
for  every  X  >0,  to  the  integral  of  a  strictly  negative 
function  over  a  set  of  positive  measure.   ■ 
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LEMMA  A. 2 ;For  k=1,2,...m,  the  following  relation  holds; 


3E  ex  (X)     ?      3E  a  X  (X) 

— =   1  X  .      ~i~ 

dX,  i=1  ^       3X, 


Proof:  Again,  without  loss  of  generality,  assume  k=m. 
Define 

m 


D   =  {a:  0<a<1 ,     7  X.a.  <  1  } 
a         —  -      .  ^ .  X  1  - 


i  =  1 


m 


D   =  {c:  0<c<1  ,    c  >   y  X .a. } 

1=1 


We  have,  by  Lemma  A.I, 


3E  ex  (X) 


3X 


m 


3X 
m 


cdF 


dF 


D 


D 


D 


dU 

dX      a 
m 


m 


y  X.a. 
^      11 


i=1 


where  U  ( X  )  = 
c   m 


m 


(A. 6) 


CdF  . 
c 


Henee,  for  X  >0, 


dU, 


m 


m 


m    dX 


m 


y  x.a. a   f  (  y  X.a.),  which, 
.^,iim   c.^,11         ' 


i=1 


i=1 


substituted  in  (A. 6),  gives 


3E  ex  (X) 


3X 


m 


m       ^  m 

yx.    /   -a.af  (yx.a. 

.^,1    /     ime  .^,11 

1=1     J  1=1 
D 


)dF 


(A. 7) 
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m-1 

Applying  the  substitution  a  = (c-  T   A.a.)/X   to  the 

m     .^    1  1    m 

integral  in  (A. 7)   and  performing      straightforward 
calculations,  we  obtain 

m 

A.  a  )df 
a 


m 

/   -a.a  f   (  y  A.a. )dF 
/      1  m  c   .^,11    c 


D 


dA       ' 
m 


(A. 8) 


where  D",  F'  and  U  are  defined  as  in  Lemma  A.I.  Combining 
(A. 7)  and  (A. 8),  the  required  result  easily  follows  for 

A  >0.  Continuity  of  the  partial  derivatives  implies  that 

m  '^ 

Lemma  A. 2  holds  also  for  A  =0.   ■ 

m 

Lemma  3.2  and  3.3  can  now  be  proved. 


Proof  of  Lemma  3.2:   For  k.=  l,2,...m,  we  have 


3L_ 
8  A, 


3E  a  x^{A)    3E  cx^ ( A ) 
+  — 


A, 


m 

I  A 
i=1 


3E  a.x  (A) 

— 1— 


=  \  - 


3E  ax  (A) 

9a! 


by  Lemmas  A.I  and  A. 2.  This  implies  that  L(A)  is  conti- 
nuously dif f erentiable  and  that  VL(A)=6-E  ax  (A). 
Moreover,  we  have 


3A 


^ 


3L 


3  A, 


3A 


(6j^-E  a^x  (A)  ) 


3E  a,  x  (A) 
— k— 

3A  . 


>  0  , 


again  by  Lemma  A.I.  This  implies  that  L(A)  is  twice 
continuously  dif ferentiable   and  strictly  convex  over 
the  region  A>0,  as  its  Hessian  matrix  H(X)  is  positive 
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definite  for  A>0. 

L(\)  is  strictly  convex,  so  that  if  it  has  minimum 
over  the  region  X>0,  that  minimum  must  be  unique.  To  see 
that  at  least  one  minimum  exists,  one  can  proceed  as  in 
the  proof  of  Lemma  3.4. 

Proof  of  Lemma  3.3;   Since  L(X)  is  continuously  differen- 
tiable,  and  the  constraints  X>0  do  satisfy  the  first- 
order  constraint  qualifications  in  X"  /4/,  Kuhn-Tucker 
conditions  for  optimality  hold  at  X"and  lead  immediately 
to  (i)  and  (ii)  . 
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